Autonomous Robotic Catheter for Minimally Invasive Interventions

ABSTRACT

A robotic system comprising a robotic catheter steerable by a motorized drive system, an imaging device positioned on a distal end of the robotic catheter, and a controller configured to: process one or more images captured by the imaging device to identify an anatomical feature, implanted device, or medical instrument; estimate a location of the imaging device in the body based on the identified anatomical feature, implanted device, or medical instrument; determine, based on the estimated location of the imaging device, a direction in which to steer the robotic catheter for advancement towards an interventional site; and monitor at least one of (i) a stream of images captured by the imaging device and (ii) a force or distance measurement captured by the imaging device or a sensor proximate the imaging device, to adjust the direction in which to steer the robotic catheter during advancement towards the interventional site.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of and priority to U.S. ProvisionalApplication No. 62/965,554, filed Jan. 24, 2020 which is herebyincorporated by reference in its entirety for all purposes.

FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

This invention was made with government support under Grant No.RO1HL124020, awarded by the National Institutes of Health (NIH). Thegovernment has certain rights in the invention.

FIELD

The present disclosure relates to robotic systems designed to performmedical procedures and, in particular, autonomous robotic catheters forminimally invasive procedures within the body, such as brain and cardiacsurgeries.

BACKGROUND

Minimally invasive surgery reduces the trauma associated withtraditional open surgery, resulting in faster recovery time, fewer woundinfections, reduced postoperative pain, and improved cosmesis. Thetrauma of open-heart surgery is particularly acute because it involvescutting and spreading the sternum to expose the heart. An importantadditional step to reducing procedural trauma and risk in cardiacprocedures is to develop ways to perform repairs without stopping theheart and placing the patient on cardiopulmonary bypass.

To this end, many specialized devices have been designed that replicatethe effects of open surgical procedures, but which can be delivered bycatheter. These include transcatheter valves, mitral valve neochords,occlusion devices, stents, and stent grafts. To deploy these devices,catheters are inserted either into the vasculature (e.g., femoral veinor artery) or, via a small incision between the ribs, directly into theheart through its apex.

From the point of insertion, the catheter must be navigated to the siteof the intervention inside the heart or its vessels. Beating-heartnavigation is particularly challenging because blood is opaque andcardiac tissue is moving. Despite the difficulties of navigation,however, the most critical part of the procedure is device deployment.This is the component when the judgment and expertise of the clinicianare most crucial. Much like the autopilot of a fighter jet, autonomousnavigation can relieve the clinician from performing challenging, butroutine, tasks so that they can focus on the mission-critical componentsof planning and performing device deployment.

To safely navigate a catheter, it is necessary to be able to determineits location inside the heart and to control the forces it applies tothe tissue. In current clinical practice, forces are largely controlledby touch, whereas catheter localization is performed using fluoroscopy.Fluoroscopy provides a projective view of the catheter, but it does notshow soft tissue and exposes the patient and clinician to radiation.Ultrasound, which enables visualization of soft tissue and catheters, isoften used during device deployment, but the images are noisy and oflimited resolution. In conjunction with heart motion, this makes itdifficult to precisely position the catheter tip with respect to thetissue.

SUMMARY

The present disclosure relates to an autonomous robotic catheter forminimally invasive interventions. One such robotic system, in variousembodiments, may comprise a robotic catheter steerable by a motorizeddrive system; an imaging device positioned on a distal end of therobotic catheter; and a controller configured to process one or moreimages captured by the imaging device to identify an anatomical feature,implanted device, or medical instrument in the one or more images,estimate a location of the imaging device in the body based on theidentified anatomical feature, implanted device, or medical instrument,determine, based on the estimated location of the imaging device, adirection in which to steer the robotic catheter for advancement towardsan interventional site, and monitor at least one of (i) a stream ofimages captured by the imaging device and (ii) force or distancemeasurements captured by the imaging device or by a sensor proximate theimaging device, to adjust the direction in which to steer the roboticcatheter during advancement towards the interventional site.

The robotic catheter, in various embodiments, may be comprised of two ormore concentric tubes. The motorized drive system, in variousembodiments, may be operable to rotate and translate the roboticcatheter.

The imaging device, in various embodiments, may include one of an imagesensor, a camera, an ultrasonic probe, or other device configured tocapture the one or more images. The controller, in various embodiments,may be configured to adjust the direction in which to steer the roboticcatheter so as to maintain constant or intermittent contact with ananatomical feature during advancement towards the interventional site.In an embodiment, the imaging device may include a surface configured todisplace bodily fluid from a contact interface between the imagingwindow and the anatomical feature, the implanted device, or the medicalinstrument. In another embodiment, the imaging device may include animaging window covering the imaging device and the imaging window mayinclude a surface configured to displace bodily fluid from a contactinterface between the imaging window and the anatomical feature,implanted device, or medical instrument.

In various embodiments, processing one or more images captured by theimaging device may include comparing the one or more of the capturedimages to representative images of one or more anatomical features,implanted devices, or medical instruments present along a pathway to ainterventional site.

In various embodiments, estimating a location of the imaging device inthe body may include identifying the location of the identifiedanatomical feature, implanted device, or medical instrument in ananatomical model. In various embodiments, determining the direction inwhich to steer the robotic catheter for advancement towards ainterventional site may include determining a vector between theestimated location of the imaging device and the interventional siteusing an anatomical model. The vector, in an embodiment, may be used inplanning a path from the estimated location to the interventional site.The anatomical model, in some embodiments, may be non-dimensional. In anembodiment, the anatomical model may be obtained from pre-procedural orintraoperative imaging.

In various embodiments, monitoring a stream of images captured by theimaging device to adjust the direction in which to steer the roboticcatheter may include identifying whether the imaging device iscontacting the anatomical feature, implanted device, or medicalinstrument based on whether at least a portion of an image of the streamof images is unobstructed by bodily fluid. In various embodiments,monitoring a force measurement to adjust the direction in which to steerthe robotic catheter may include determining whether the forcemeasurement is substantially non-zero.

The controller, in various embodiments, may be configured to estimate acontact force between the imaging device and the anatomical featurebased on how much of the one or more images is unobstructed by bodilyfluid. In an embodiment, the controller may be configured to use theestimated contact force or the force measurement to avoid generatingunsafe contact forces between the imaging device and the anatomicalfeature, the implanted device, or the medical instrument. In anembodiment, the controller may be configured to estimate an orientationof the imaging device relative to the anatomical feature based on adistribution of the contacting surface area with respect to a center ofthe image.

In various embodiments, monitoring a distance measurement to adjust thedirection in which to steer the robotic catheter may include determininga distance to the anatomical feature, implanted device, or medicalinstrument. The controller may be configured to adjust the direction inwhich to steer the robotic catheter so as to avoid contact with ananatomical feature during advancement towards the interventional site.

In another aspect, the present disclosure is directed to a roboticsystem comprising a catheter; an imaging device positioned on the distalend of the catheter; and a processor configured to process one or moreimages captured by the imaging device to automatically perform at leastone of the following: identify an anatomical feature, implanted device,or medical instrument contacted by the imaging device by comparing theone or more captured images to representative images of one or moreanatomical features, implanted devices, or medical instruments presentalong a pathway to an interventional site, and estimate a contact forcebetween the imaging device and the anatomical feature, implanted device,or medical instrument based on how much of the one or more images isunobstructed by bodily fluid.

In various embodiments, the catheter may be manually steered by aperson. The processor, in various embodiments, may be configured toestimate a location of the imaging device in the body based on theidentified anatomical feature, implanted device, or medical instrument.In various embodiments, estimating a location of the imaging device inthe body based on the identified anatomical feature may includeidentifying the location of the identified anatomical feature, implanteddevice, or medical instrument in an anatomical model.

In various embodiments, the processor may be configured to determine,based on the estimated location of the imaging device, a direction inwhich to steer the catheter for advancement towards an interventionalsite. In an embodiment, determining the direction in which to steer thecatheter for advancement towards a interventional site may includedetermining a vector between the estimated location of the imagingdevice and the interventional site using an anatomical model. Thevector, in an embodiment, may be used in planning a path from theestimated location to the interventional site. The anatomical model, invarious embodiments, may be non-dimensional. In an embodiment, theanatomical model may be obtained from pre-procedural or intraoperativeimaging.

In various embodiments, the processor may be configured to use theestimated contact force to avoid generating unsafe contact forcesbetween the imaging device and the anatomical feature, implanted device,or medical instrument. Additionally or alternatively, the processor, inan embodiment, may be configured to display, on a display device, one ormore of the one or more captured images, the estimated location of theimaging device, the direction in which to steer the catheter foradvancement towards the interventional site, and the estimated contactforce.

BRIEF DESCRIPTION OF THE DRAWINGS

Illustrative, non-limiting example embodiments will be more clearlyunderstood from the following detailed description taken in conjunctionwith the accompanying drawings.

FIG. 1A and FIG. 1B illustrate a representative transjugular roboticcatheter;

FIG. 2 depicts a representative embodiment of a robotic system includinga support structure, one or more robotic arm modules, one or more toolmodules, and a drive system;

FIG. 3 shows a drive system mounted on the operating room table using apassively adjustable frame that allows the catheter tip to be positionedand oriented for entry into the apex;

FIG. 4 illustrates a 3D model of an adult human left ventricle;

FIG. 5 is a table showing the tube parameters of a representativerobotic catheter;

FIG. 6 depicts a representative embodiment of the haptic vision sensor;

FIG. 7 illustrates a representative embodiment of the haptic visionsensor next to a penny for scale;

FIG. 8 shows the robotic catheter tip pressed laterally against cardiactissue;

FIG. 9 depicts the robotic catheter being operated in an intermittentcontact mode in which the catheter is in contact with the heart wall fora specified fraction of the cardiac period;

FIG. 10 illustrates the complex navigational tasks that can be achievedby following a path through a connectivity graph;

FIG. 11 illustrates a current clinical approach to paravalvular leakclosure;

FIG. 12 depicts the valve clock coordinates;

FIG. 13 depicts a vascular occluder used to plug leaks;

FIG. 14A, FIG. 14B, FIG. 14C, and FIG. 14D depict graphs illustratingnavigation completion times from in vivo experiments;

FIG. 15 illustrates a representative software development cycle;

FIG. 16 depicts a representative occluder deployment system;

FIG. 17A illustrates a handheld instrument for the simultaneousmeasurement of tip contact force and contact duty cycle that combines ahaptic vision sensor with a force sensor;

FIG. 17B shows a graph illustrating in vivo measurements of the temporalvariations in contact force as a function of contact duty cycle on theaortic valve annulus; and

FIG. 17C shows a graph illustrating maximum contact force measurementsas a function of contact duty cycle.

DETAILED DESCRIPTION

Embodiments of the present disclosure are directed to robotic systemsfor performing medical procedures and, in particular, minimally invasiveprocedures within the body such as brain and cardiac surgeries. Therobotic systems described herein may be configured to autonomouslynavigate a robotic arm to an interventional site within the body. Theuse of autonomy can be applied in many catheter procedures. In the longterm, autonomy may enable a senior clinician to safely oversee robotsperforming procedures on multiple patients simultaneously (concurrentsurgery). In the shorter term, autonomous catheters will act as expertassistants to clinicians guiding them through both the routine anddifficult parts of a procedure.

Embodiments of the present disclosure generally include a robotic arm orcatheter and software that combines inputs from real-time andpreoperative sensing together with a database including anatomical mapsand machine learning algorithms that encode best practices forperforming repairs. The robotic systems make procedures easier forclinicians by providing improved sensing (3D positioning and cathetershape visualization inside, e.g., the heart/vasculature) usingmulti-modal sensing including integrated optical, ultrasound, OCT, andforce sensing, external sensing such as ultrasound, and intuitivecatheter/device motion control. The robotic systems may be designed tosend and receive data and algorithms from other robotic catheter devicesand from other centralized data centers.

The robotic system incorporates procedural planning capabilities and,based on preoperative studies, such as imaging, proposes to theclinician an optimized set of repairs based on the available app's andhardware, e.g., to repair mitral regurgitation, it could proposeannuloplasty together with a set of neochords (number, location onleaflets and attachment locations near or on papillary muscles). Theclinician can accept these repairs and/or modify them such as byspecifying alternate repairs and repair locations through the graphicalinterface. Once the clinician approves the overall plan, the roboticsystem would propose plans for each subtask on a graphical interface.These plans would be based on the robotic system's current autonomousmotion algorithms. The clinician would have the option of approving ormodifying the task motions. Once satisfied with a motion, the cliniciancan have the robot perform it autonomously or the operator can performit via teleoperation. If by teleoperation, the robotic system canprovide feedback, e.g., graphical and force feed long the desired path.This process would continue until all tasks are completed.

Various embodiments of the present disclosure may employ wall-followingautonomous navigation techniques, as later described in more detailherein. In nature, wall following—tracing object boundaries in one'senvironment—is used by certain insects and vertebrates as an exploratorymechanism in low-visibility conditions to ameliorate their localizationand navigational capabilities in the absence of visual stimuli.Positively thigmotactic animals, which attempt to preserve contact withtheir surroundings, use wall following in unknown environments as anincremental map-building function to construct a spatial representationof the environment. Animals initially localize new objects found bytouch in an egocentric manner, i.e., the object's relative position tothe animal is estimated; however, later, more complex spatial relationscan be learned, functionally resembling a map representation. Theseanimals often sample their environment by generating contact such asthrough rhythmically controlled whisker motion, as reported in rodents,or antennae manipulations in cockroaches and blind crayfish.

Such techniques may be applicable to many minimally invasive proceduresincluding those in the vasculature, airways, gastrointestinal tract andthe ventricular system of the brain. The catheter acts as a platformtechnology which can be adapted to new procedures or to the delivery ofnew devices by adding software/hardware applications. For example, adevice for structural heart repair may offer software/hardwareapplications for mitral annuloplasty and for neochord placement. Thedevice could also offer multiple software/hardware applications for thedelivery of equivalent devices offered by different manufacturers, e.g.,mitral clips. Besides structural heart repair, software/hardwareapplications can be provided for diagnostic angiograms, coronary stentdelivery and aortic stent graft placement.

From a safety and learning standpoint, the robotic system's conditionwill be monitored in real-time by software that can identify and reactto contingencies like hardware failures; either by shifting toteleoperation mode or by performing a pre-approved safety action (e.g.,park at a safe location until the medical team provides newinstructions).

The robotic system may collect data including (1) preoperative studydata that was used for planning, (2) proposed and approved surgicalplan, (3) input and output sensor data used by the robot, (4) operatorcommands, (5) robot planning and navigation algorithms used, and (6)procedure outcome data including, e.g., imaging studies. This data isused to refine both parametric and nonparametric models/algorithms ofboth the robotic system by itself and of the robot performing thespecific procedure. Models and algorithms can be classified on the basisof a particular type of repair or more specifically, e.g., by aparticular anatomical variation. The refined models and algorithms arethen evaluated for safety following appropriate established protocols.Once approved, the models and algorithms on the robotic platform areupdated for that procedure.

Model and algorithm refinement can be performed using only local datafrom a single robotic system. Alternately, it can pool data from acollection of robotic systems at, e.g., a single hospital, or from alarge number of installed systems. In this way, the knowledge gainedfrom every procedure performed is made available to all users andpatients.

To show that autonomous navigation is possible, we investigated it inthe hardest place to do it—inside the beating heart. FIG. 1A and FIG. 1Billustrate a representative transjugular robotic catheter of the presentdisclosure. In particular, FIG. 1A depicts the catheter passing throughvasculature to the right atrium and FIG. 1B illustrates how the designof telescoping robotic catheter modules allows the robotic catheter tobe steered to and about the interventional site.

We created a robotic catheter that can navigate through the blood-filledheart using wall-following algorithms inspired by positivelythigmotactic animals. The catheter uses haptic vision, a hybrid senseusing imaging for both touch-based surface identification and forcesensing, to accomplish wall following inside the blood-filled heart.Through in vivo animal experiments, we demonstrate that the performanceof an autonomously controlled robotic catheter rivaled that of anexperienced clinician. Autonomous navigation is a fundamental capabilityon which more sophisticated levels of autonomy can be built, e.g., toperform a procedure. Similar to the role of automation in a fighteraircraft, such capabilities can free the clinician to focus on the mostcritical aspects of the procedure while providing precise and repeatabletool motions independent of operator experience and fatigue.

Inspired by this approach, we designed positively thigmotacticalgorithms that achieve autonomous navigation inside the heart bycreating low-force contact with the heart tissue and then followingtissue walls to reach a goal location. To enable wall following whilealso locally recapturing the detailed visual features of open surgery,we introduced a sensing modality at the catheter tip that we call“haptic vision.” Haptic vision combines intracardiac endoscopy, machinelearning, and image processing algorithms to form a hybrid imaging andtouch sensor—providing clear images of whatever the catheter tip istouching while also identifying what it is touching (e.g., blood,tissue, and valve) and how hard it is pressing.

Our primary result is that autonomous navigation in minimally invasiveprocedures is possible and can be successfully implemented usingenhanced sensing and control techniques to provide results comparablewith expert manual navigation in terms of procedure time and efficacy.Furthermore, our experiments comparing clinician-controlled roboticnavigation with manual navigation echo the results obtained for manymedical procedures—robots operated by humans often provide no betterperformance than manual procedures except for the most difficult casesand demanding procedures. Medical robot autonomy provides an alternativeapproach and represents the way forward for the field.

Automating such tasks as navigation can provide important benefits toclinicians. For example, when a clinician is first learning a procedure,a significant fraction of their attention is allocated to controllinginstruments (e.g., catheters and tools) based on multimodal imaging.Once a clinician has performed a large number of similar procedures withthe same instruments, the amount of attention devoted to instrumentcontrol is reduced. By using autonomy to relieve the clinician ofinstrument control and navigation, the learning curve involved inmastering a new procedure could be substantially reduced. This would beof significant benefit during initial clinical training, and it may alsoenable midcareer clinicians to adopt new minimally invasive techniquesthat would otherwise require too much retraining. In addition, evenafter a procedure is mastered, there are many situations where anindividual clinician may not perform a sufficient number of proceduresto maintain mastery of it. In all of these cases, autonomy could enableclinicians to operate as experts with reduced experience- andfatigue-based variability.

There are also many places in the world where clinical specialties arenot represented. Although medical robots can provide the capability fora specialist to perform surgery remotely, this approach requiresdedicated high-bandwidth two-way data transfer. Transmission delays orinterruptions compromise safety owing to loss of robot control. In thesesituations, autonomy may enable stable and safe robot operation evenunder conditions of low-bandwidth or intermittent communication.Autonomy may also enable the robot to detect and correct for changingpatient conditions when communication delays preclude sufficiently fastreaction by the clinician.

Autonomy also enables, to an unprecedented degree, the capability toshare, pool, and learn from clinical data. With teleoperated robots,robot motion data can be easily collected, but motions are beingperformed by clinicians using different strategies, and the informationthey are using to guide these strategies may not all be known, let alonerecorded. In contrast, the sensor data streaming to an autonomouscontroller are well defined, as is its control strategy. Thiscombination of well-defined input and output data, together with knowncontrol strategies, will make it possible to standardize and improveautonomous technique based on large numbers of procedural outcomes. Inthis way, robot autonomy can evolve by applying the cumulativeexperiential knowledge of its robotic peers to each procedure.

Robotic System 100

FIG. 2 depicts a representative embodiment of robotic system 100.Robotic system 100 of the present disclosure may generally include asupport structure 200, one or more robotic arm modules 300, one or moretool modules 400, and a drive system 500. In various embodiments, eachrobotic arm module 300 and tool module 400 may detachably couple tosupport structure 200 and are operated by drive system 500 to perform asurgical procedure. Robotic arm module(s) 300 may include a robotic arm310 having working channel 311 (not shown) for accommodating an elongatetool 410 of tool module 400 and robotic arm 310 can be steered about ainterventional site to position and orient a distal end of elongate tool410 during the procedure. As configured, elongate tool 410, in variousembodiments, may be removed from working channel 311 of robotic arm 310during the procedure without having to withdraw robotic arm module 300from the interventional site, thereby allowing tool modules 400 to beswapped in and out during a procedure and medical devices (e.g., stents)to be implanted through working channel 311 with ease and speed.Components of robotic system 100, in various embodiments, may beanalogous to those of the robotic device described in U.S. ProvisionalPatent Application No. 62/965,399 filed Jan. 24, 2020, which isincorporated by reference herein in its entirety.

For the specific autonomous navigation experiments described below, wedesigned the catheter using concentric tube robot technology in whichrobots are composed of multiple needle-sized concentrically combinedprecurved superelastic tubes. A motorized drive system located at thebase of the tubes rotated and telescopically extended the tubes withrespect to each other to control the shape of the catheter and its tipposition. The drive system was mounted on the operating room table usinga passively adjustable frame that allowed the catheter tip to bepositioned and oriented for entry into the apex (FIG. 3).

Tools and devices were delivered through the lumen of the innermostrobot tube, which incorporated a valve and flush system at its proximalend. This system enabled the catheter lumen to be flushed with saline toprevent air entry into the heart and to prevent pressurized blood fromthe heart from entering the lumen of the catheter. We used a designoptimization algorithm to solve for the tube parameters based on theanatomical constraints and clinical task (e.g., aortic paravalvular leakclosure). The anatomical and task constraints were defined using a 3Dmodel of an adult human left ventricle (FIG. 4). Because the relativedimensions of the human and porcine hearts are similar, the resultingdesign was appropriate for our in vivo experiments. The design algorithmsolved for the tube parameters enabling the catheter tip to reach fromthe apex of the heart to a set of 25 uniformly sampled points around theaortic valve annulus without the catheter contacting the ventricularwall along its length. The orientation of the catheter tip was furtherconstrained at the 25 points to be within 10° of orthogonal to the valveplane. The resulting design was composed of three precurved superelastictubes forming two telescoping sections with motion as shown in FIG. 4.The tube parameters of the robotic catheter are given in FIG. 5.

Haptic Vision Sensor 410

FIG. 6 depicts a representative embodiment of haptic vision sensor 410of the present disclosure. Haptic vision sensor 410, in variousembodiments, may generally comprise an imaging device 411 such as animage sensor (e.g., charge-coupled device (CDD) or complementary metaloxide semiconductor (CMOS)), a camera, an ultrasonic probe, or otherdevice configured to capture one or more images. Sensor 410, in variousembodiments, may optionally include a light source 412 and/or an imagingwindow 413 covering imaging device 411. Imaging window 413, in variousembodiments, may be transparent and positioned in front of imagingdevice 411 to seal off a space in front of imaging device 411 fromsurrounding bodily fluids for improved visibility, much like asnorkeling mask. Together, these components combine to define a field ofview 414 of imaging device 411.

As later described in more detail, robotic system 100 may employwall-following techniques to autonomously navigate to a interventionalsite in the body. Haptic vision sensor 410, in various embodiments, maycontinuously or intermittently contact tissue and other anatomicalfeatures within the body to assess where it is and which direction to gonext. When haptic vision sensor 410 is pressed against the tissue, itdisplaces blood or other bodily fluids obstructing the field of view 414of imaging device 411. Generally speaking, higher contact forcedisplaces more fluid from the contact interface between imaging window413 and the tissue or displaces fluid over a greater portion of thecardiac cycle, thereby increasing the amount of tissue visible toimaging device 411, and vice versa. Accordingly, in various embodiments,the amount of tissue visible in the field of view 414 of imaging device411 may serve as a proxy for estimating contact force.

In various embodiments, robotic system 100 may be configured to analyzeimages captured by imaging device 411 to determine a location of hapticvision sensor 410 within the body. As later described in more detail,robotic system 100 may use any number of suitable image processingtechniques to compare a captured image with images stored in a databaseof various tissues and anatomical features to identify a likely matchand thereby determine a location of haptic vision sensor 410 within thebody. Accordingly, haptic vision sensor 410 may act as a combined sensorfor detecting contact, estimating contact force, and determining alocation of haptic vision sensor 410 within the body based on imagescaptured by imaging device 411.

Still referring to FIG. 6, robotic system 100, in various embodiments,may further comprise a working channel 415 through which a tool or otherdevice may be advanced to the interventional site. As configured, toolmodule 400 need not necessarily be removed from robotic arm 310 in orderto make room for advancing the tool or other device to theinterventional site through working channel 311 of robotic arm 310.Instead, haptic vision sensor 410 may remain in place when using thetool and advantageously allow an operator (e.g., surgeon) to see thetool and/or interventional site via imaging device 411 during theprocedure. Additionally or alternatively, robotic system 100, in variousembodiments, may further comprise an electromagnetic tracker 416. Aslater described in more detail, the kinematic model of the catheter maybe used to determine the location of the catheter tip in space. Sincethe catheter deforms owing to contact with the anatomy, however, thisestimate can be inaccurate. A tip-mounted tracking sensor or a cathetershape sensor that computes the actual shape of the catheter can be usedto provide more accurate measurements of tip location. Thesemeasurements can be used to build an anatomical map as described laterin the application (valve annulus model). They can also be used togetherwith imaging-based anatomic feature identification to localize thecatheter tip with respect to an anatomic model, perform registration ofthe anatomic model with respect to the robot and the patient, to scalethe anatomic model with respect to the patient and to guide motiontoward the procedural target.

FIG. 7 shows a representative embodiment of haptic vision sensor 410next to a penny for scale. We fabricated this prototype of sensor 410using a 1-mm³ complementary metal-oxide semiconductor imaging device 411(NanEye Camera System, AWAIBA) and a 1.6 mm by 1.6 mm light-emittingdiode 412 (XQ-B LED, Cree Inc.) encased in an 8-mm-diameter siliconeimaging window 414 (QSil 218, Quantum Silicones LLC.) molded onto astainless-steel body. The diameter of imaging window 414 was selected toprovide a field of view 414 facilitating both autonomous andoperator-guided navigation. Sensor 410 also incorporated a2.5-mm-diameter working channel 415 for device delivery. Sensor 410 isclamped to the tip of the catheter 402 such that the lumen of theinnermost catheter tube is aligned with working channel 415. Although wehave also designed catheters 402 in which the sensor wiring was runthrough the catheter lumen, in these experiments, the wiring for theimaging device 411 and LED 412 were run outside so that sensor 410 couldbe replaced without disassembling robotic system 100. Note that FIG. 2shows an alternate design in which the cardioscope is designed to passthrough the robotic catheter.

Wall-Following Autonomous Navigation

As previously noted, robotic system 100 may employ wall-followingtechniques to autonomously navigate to a interventional site in thebody. Robotic system 100, in various embodiments, may include acontroller configured to steer the robotic catheter in a manner thatcauses it to continuously or intermittently contact anatomical featureswithin the body to assess where it is and which direction to go next inorder to reach the interventional site. In various embodiments,controller may utilize information captured or collected by hapticvision sensor 410 to this and related ends, as described in more detailbelow.

The controller, in various embodiments, may be configured to process oneor more images captured by imaging device 411 to identify an anatomicalfeature contacted by imaging device 411. As used herein, the term“anatomical feature” broadly refers to solid anatomical features liketissues, organs, and the like (as opposed to bodily fluids) that may bepresent along a pathway (e.g., one defined by the vasculature) to theinterventional site. As described in more detail below, the controllermay be configured to compare the one or more of the images captured byimaging device 411 to representative images of anatomical featurespresent along a pathway to a interventional site. The representativeimages may be stored in a database accessed by the controller and thecontroller may parse through the images until it identifies a likelymatch with the captured image. Machine learning techniques may be usedin this process as later described in more detail. Upon identifying amatch, the controller may determine where the identified anatomicalfeature is in a representative anatomical model and thus understandwhere imaging device 411 is in the body. The controller may use similartechniques to identify when it has reached a medical device/instrumentor implant (e.g., a prosthetic valve) in the body. In variousembodiments, the controller may be configured to steer the system suchthat it follows a surface(s) of the medical device/instrument or implantsimilar to the way in which it may follow an anatomical feature/tissueto the interventional site.

The controller, in various embodiments, may also use the estimatedlocation of imaging device 411 along with the anatomical model todetermine a direction in which to steer the robotic catheter foradvancement towards a particular interventional site. In particular, thecontroller may be configured to identify a suitable pathway in theanatomical model (e.g., a particular artery or vein, or pathway throughthe chambers of the heart) that extends between the estimated locationof imaging sensor 411 and the interventional site, and thereby determinea direction in which to direct the robotic catheter to follow thatpathway generally. In an embodiment, the anatomical model isnon-dimensional, thereby allowing it to be used with persons of varyingshapes and sizes.

The controller, in various embodiments, may use tip tracking sensors orcatheter shape sensors to track its motion and use the data to validateand refine its estimated position and orientation with respect to ananatomical model. It may also use this data to adjust the size and shapeof a dimensionless anatomical model to fit that of a patient. It canalso use this data to create an anatomical map as described later in theapplication. Furthermore, it can use the motion data to guide itsadvancement toward a procedural site.

The controller, in an embodiment, may be further configured to estimatean orientation of imaging device 411 relative to an anatomical featureand use this as an additional input to facilitate steering the roboticcatheter. In particular, if the feature is locally planar, the surfacenormal to the plane may be estimated from the images. If, furthermore,the sensor is in contact with the feature, the distribution of thecontacting surface area with respect to the center of the image can beused to estimate the relative orientation.

The controller, in various embodiments, may adjust the direction inwhich the robotic catheter is being steered in order to maintainintermittent or continuous contact with surrounding anatomical featuresduring advancement towards the interventional site as part of awall-following navigation approach. In particular, in variousembodiments, controller may monitor at least one of (i) a stream ofimages captured by imaging device 411, and (ii) a force measurementcaptured by a sensor proximate the imaging device, to determine whetherimage sensor 411 is in contact with an anatomical feature. With respectto (i), as previously described, when image sensor 411 contacts ananatomical feature, bodily fluid is displaced from the contact interfacesuch that all or a portion of the image contains an unobstructed view ofthe anatomical feature. Accordingly, if all or a portion of a givenimage is unobstructed by bodily fluid, the controller may determine thatimaging device 411 is in contact with an anatomical feature. Conversely,if the entire image is obstructed by bodily fluid, the controller maydetermine that imaging device 411 is not in contact with an anatomicalfeature. Regarding (ii), the controller may determine that image sensor411 is in contact with an anatomical feature if the force measurement issubstantially non-zero and not in contact with an anatomical feature ifthe force measurement is substantially zero.

The controller, in various embodiments, may use information aboutwhether imaging sensor 411 is in contact with the anatomical feature toadjust the direction in which to steer the robotic catheter inaccordance with a particular wall-following technique being utilized.For example, in embodiments employing continuous contact wall followingtechniques, the controller may adjust the direction of the roboticcatheter back towards the anatomical feature (or towards an upcominganatomical feature, as predicted from the anatomical map or otherwise)if the controller detects that imaging device 411 is no longer incontact with the anatomical feature. As another example, in embodimentsemploying intermittent contact wall following techniques, the controllermay adjust the direction of the robotic catheter back towards theanatomical feature after a predetermined period of time or distancetravelled to again establish contact with the anatomical feature, whereit can again estimate the location of imaging sensor 411 and therebyconfirm whether adjustments to the general steering direction need to bemade to successfully navigate to the interventional site.

The controller, in various embodiments, may additionally oralternatively estimate the associated contact force. When imaging device411 is pressed against the tissue, it displaces blood or other bodilyfluids obstructing the field of view 414 of imaging device 411.Generally speaking, higher contact forces displace more fluid from thecontact interface, thereby increasing the portion of the image in whichthe anatomical feature is visible and unobstructed by bodily fluid, andvice versa. Accordingly, in various embodiments, the size of anunobstructed portion of an image may serve as a proxy for estimatingcontact force. In another embodiment, the aforementioned force sensormay additionally or alternatively provide force measurements indicatingthe contact force. The controller may use the estimated contact force orthe force measurement to avoid generating unsafe contact forces betweenimaging device 411 and an anatomical feature. For example, thecontroller may cease advancing the robotic catheter in a currentdirection if the estimated or measured contact force exceeds apredetermined threshold for safety. It should be recognized that suchthresholds may vary based on the particular anatomical featureidentified as being contacted.

It should be recognized that while the present disclosure describesautonomous robotic navigation techniques, robotic system 100 may beadapted to facilitate manual steering of a catheter. Such a manualsystem may comprise a traditional catheter or a manually-steered roboticcatheter (e.g., manually steerable by joystick, as opposed toautonomously steered by the controller), an imaging device 411positioned on the distal end of catheter 402, and a processor. Generallyspeaking, the processor may perform analytical functions similar tothose of the controller, but rather than automatically steering thecatheter, the processor may display or otherwise provide information toa person steering the catheter to facilitate manual steering. Forexample, the processor may be configured to display on a display devicein the operating room one or more of the navigation-related parametersdescribed above—that is, the estimated location and orientation ofimaging device 411, a suggested direction in which to steer thecatheter, suggested adjustments to the direction to reestablish ormaintain contact with surrounding anatomy depending on whetherintermittent or continuous wall-following techniques are to be employed,etc. derived as explained above. Such a system could also providewarnings to the operator if contact forces (estimated or measured) mayexceed safety thresholds.

Navigation Experiment

In an experiment, we used haptic vision as the sole sensory input to ournavigation algorithms to achieve wall following while also controllingthe forces applied by the catheter tip to the tissue. We evaluatedautonomous navigation through in vivo experiments and compared it withoperator-controlled robot motion and with manual navigation. For wallfollowing, we exploited the inherent compliance of the catheter toimplement two control modes based on continuous and intermittentcontact. Continuous contact can often be safely maintained over thecardiac cycle when the catheter tip is pressed laterally against thetissue because catheters are highly compliant in this direction (FIG.8). Intermittent contact can be necessary when there is substantialtissue motion and the catheter is against the tissue along its stifferlongitudinal axis (FIG. 9).

To perform wall following, we designed a machine learning-based imageclassifier that can distinguish between blood (no contact), ventricularwall tissue, and the bioprosthetic aortic valve. The algorithm used thebag-of-words approach to separate images into groups (classes) based onthe number of occurrences of specific features of interest. Duringtraining, the algorithm determined which features were of interest andthe relationship between their number and the image class. For training,we used OpenCV to detect features in a set of manually labeled trainingimages. Next, the detected features were encoded mathematically usingLUCID descriptors for efficient online computation. To reduce the numberof features, we identified the optimal feature representatives usingclustering (k-means). The resulting cluster centers were therepresentative features used for the rest of the training, as well asfor runtime image classification. Having identified the set ofrepresentative features, we made a second pass through the training datato build a feature histogram for each image by counting how many timeseach representative feature appeared in the image. The final step was totrain a support vector machine (SVM) classifier that learned therelationship between the feature histogram and the corresponding class.

Using the trained algorithm, image classification proceeded by firstdetecting features and computing the corresponding LUCID descriptors.The features were then matched to the closest representative features,and the resulting feature histogram was constructed. On the basis of thehistogram, the SVM classifier predicted the tissue-based contact state.We achieved good results using a small set of training images (˜2000images) with training taking ˜4 min. Because image classification took 1ms, our haptic vision system estimated contact state at the frame rateof the camera (45 frames/s). The contact classification algorithm wasaccurate 97% of the time (tested on 7000 images not used for training)with type I error (false positive) of 3.7% and type II (false negative)of 2.3%.

In both the continuous and the intermittent contact modes, the robotacted to limit the maximum force applied to the tissue using a hapticvision-based proxy for force. In the continuous contact mode, catheterposition with respect to the tissue surface was adjusted to maintain aspecified contact area on the catheter tip (FIG. 8) corresponding to adesired force. In the intermittent contact mode, catheter position withrespect to the tissue surface was adjusted to maintain a desired contactduty cycle—the fraction of the cardiac cycle during which the catheterwas in tissue contact (FIG. 9). The relationship between contact dutycycle and maximum force was investigated experimentally as described inthe section below titled “In-vivo calibration of contact duty cycleversus maximum tissue force.” Complex navigation tasks can be achievedby following a path through a connectivity graph (FIG. 10) and selectingbetween continuous and intermittent contact modes along that path basedon contact compliance and the amplitude of tissue motion.

FIG. 8 depicts the robotic catheter 100 being operated in a continuouscontact mode in which catheter tip is pressed laterally against heartwall over entire cardiac motion cycle. Contact force is controlled onthe basis of the amount of tissue visible at the edge of imaging windowas shown in inset.

When the catheter was positioned laterally against cardiac tissue, itsflexibility could enable continuous contact to be maintained withoutapplying excessive force to the tissue. We used haptic vision to controlthe amount of tissue contact by controlling catheter motion in thedirection orthogonal to the tissue surface. Catheter motion in the planeof the tissue surface was independently controlled so as to produce wallfollowing at the desired velocity and in the desired direction. Thecontroller was initialized with an estimate of wall location so that ifit was not initially in tissue contact, it moved toward the wall togenerate contact. This occurred in our in vivo experiments duringnavigation from the apex to the aortic valve. The catheter started inthe center of the apex with the haptic vision sensor detecting onlyblood. It would then move in the direction of the desired wall (FIG. 10,a→b), specified using valve clock coordinates (FIG. 12), to establishcontact and then follow that wall until it reached the valve.

When the haptic vision sensor 410 was pressed laterally against thetissue, the tissue deformed around the sensor tip so that it covered aportion of the field of view 414 (FIG. 8). The navigation algorithmadjusted the catheter position orthogonal to the tissue surface tomaintain the centroid of the tissue contact area within a desired rangeon the periphery of the image, typically 30 to 40%. We implementedtissue segmentation by first applying a Gaussian filter to reduce thelevel of noise in the image. Next, we segmented the tissue using colorthresholding on the hue saturation value (HSV) and the CIE L*a*b*representations of color images. The result was a binary image, wherewhite pixels indicated tissue. Last, we performed a morphologicalopening operation to remove noise from the binary image. Aftersegmentation, the tissue centroid was computed and sent to the motioncontrol computer. Image processing speed was sufficient to provideupdates at the camera frame rate (45 frames/s).

FIG. 9 depicts the robotic catheter 100 being operated in anintermittent contact mode in which catheter is in contact with heartwall for a specified fraction, D, of the cardiac period (contact dutycycle). Insets show corresponding haptic vision images in and out ofcontact. Maximum contact force relates to contact duty cycle, D, asshown on plot and is controlled by small catheter displacementsorthogonal to the heart wall.

When a catheter is stiff along its longitudinal axis and positionedorthogonal to a tissue surface that moved significantly in the directionof this axis over the cardiac cycle, the contact forces can becomesufficiently high so as to result in tissue damage or puncture. Tomaintain contact forces at safe levels, one approach is to design thecatheter so that it can perform high-velocity trajectories that move therobotic catheter tip in synchrony with the tissue. We used analternative technique requiring only slow catheter motion so as toposition the tip such that it was in contact with the tissue for aspecified fraction of the cardiac cycle, the contact duty cycle, D (FIG.9). As described in the section below titled “In-vivo calibration ofcontact duty cycle versus maximum tissue force,” the contact duty cyclewas linearly related to the maximum contact force. The intermittentcontact mode was used during navigation around the aortic valve annulus.

We implemented intermittent contact navigation using haptic vision todetect tissue contact and, combined with heart rate data, to compute thecontact duty cycle at the frame rate of the camera (45 frames/s). Weimplemented a controller that adjusted catheter position along itslongitudinal axis to drive the contact duty cycle to the desired value.Catheter motion in the plane of the tissue surface was performed eitherautonomously or by the operator (shared control mode). In the autonomousmode, catheter motion in the tissue plane was performed only during thefraction of the cardiac cycle when the haptic vision sensor indicatedthat the catheter was not touching tissue. This reduced the occurrenceof the catheter tip sticking to the tissue surface during wallfollowing.

Example 1—Autonomous Navigation On Prosthetic Aortic Heart Valve Annulus

Intermittent contact control was used to control catheter motionorthogonal to the plane of the annulus. The desired value of contactduty cycle was typically set to be ˜40%. Thus, 40% of the cardiac cyclewas available for image processing (during contact), whereas the motionin the plane of the annulus was performed during the 60% noncontactportion of the cardiac cycle. During contact, the robot detected theblue tangent sutures on the valve (FIG. 12) using color thresholding inthe HSV color space and computed the centroid of the detected sutures.Next, a Hough transform on the thresholded image was used to estimatethe tangent of the aortic annulus. During the noncontact portion of thecardiac cycle, the algorithm generated independent motion commands inthe radial and tangential directions. In the radial direction, thecatheter adjusted its position such that the centroid of the detectedsutures was centered in the imaging frame. Motion in the tangentialdirection was performed at a specified velocity. While navigating aroundthe valve, the robot incrementally built a map of the location of theannulus in 3D space based on the centroids of the detected sutures andthe catheter tip coordinates as computed using the robot kinematicmodel. The model was initialized with the known valve diameter and thespecified direction of approach. By comparing the current tangent withthe model/map, the robot estimated its clock position on the annulus.Although not implemented, this model may also be used to estimate thevalve tangent and radial position in situations where the sutures arenot well detected.

To autonomously move to a prespecified location on the valve, it isnecessary to know how the valve is rotated with respect to the cathetertip. In the ventricular view of the valve annulus provided by hapticvision, such features are hidden. Although the model built duringannular navigation defines the coordinates of the annulus circle in 3Dspace, there was no means to refine the initial estimate of where 12o'clock fell on the circle, i.e., to establish the orientation of thevalve about its axis. To enable the robot to refine its orientationestimate, we introduced registration features into the annulus composedof green sutures located at 4, 8, and 12 o'clock. During annularnavigation, whenever the robot detected one of these features, itcompared its actual location with the current prediction of the modeland updated its estimate of valve rotation accordingly.

In clinical use, the sutures would remain visible for several monthsbefore endothelialization. Thus, they could be used for autonomousrepair of paravalvular leaks (as described below) that occur at the timeof valve implantation or soon after, as is the case for transcathetervalves.

Example 2—Autonomous Navigation for Paravalvular Leak Closure

Paravalvular leaks occur when a gap opens between the native valveannulus and the prosthetic valve. FIG. 11 illustrates a current clinicalapproach to paravalvular leak closure. Transcatheter leak closureinvolves sequentially navigating a catheter to the leak (image 1),passing a wire from the catheter through the gap (image 2), and thendeploying an expanding occluder device inside the gap (image 3). Thisprocedure is currently manually performed using multimodal imaging(electrocardiogram-gated computed tomographic angiography, transthoracicand transesophageal echo preoperatively, and echocardiography andfluoroscopy intraoperatively), takes many hours, and requires 29.9±24.5min of fluoroscopic x-ray exposure. Although transapical access isillustrated, approaching the valve from the aorta via transfemoralaccess is common.

Robotic system 100 of the present disclosure may overcome thesedisadvantages. In particular, as shown in FIG. 10 and FIG. 3, wedesigned a robotic catheter for entering through the apex of the heartinto the left ventricle, navigating to the aortic valve and deploying anoccluder into the site of a leak. A graphical interface may displaycatheter tip view and geometric model of robot and valve annulus. Usingleak locations determined from preoperative imaging, the catheter couldeither navigate autonomously to that location, or the clinician couldguide it there (FIG. 3). Occluder deployment was performed underoperator control.

As shown in FIG. 12, for testing we created a porcine paravalvular leakmodel by replacing the native aortic valve with a bioprosthetic valveincorporating three leaks. FIG. 12 includes two views of bioprostheticaortic valve designed to produce three paravalvular leaks at 2, 6, and10 o'clock positions. Blue sutures are used to detect tangent toannulus. Green sutures are used to estimate valve rotation with respectto the robotic system 100. FIG. 13 shows a vascular occluder (AMPLATZERVascular Plug II, St. Jude Medical, Saint Paul, Minn.) used to plugleaks.

We have implemented autonomous navigation based solely on haptic visionsensing and demonstrated the potential of the approach in the context ofa challenging beating-heart procedure, aortic paravalvular leak closure.During autonomous catheter navigation to the leak location, bothcontinuous and intermittent contact modes were used (FIG. 10). Fornavigation from the heart's apex to the aortic valve, the robot firstlocated the ventricular wall (a→b) and then followed it to the aorticvalve using the continuous contact mode (b→c). Because the valve annulusdisplaces by several centimeters over the cardiac cycle along the axisof the catheter, the robot switched to the intermittent contact modeonce it detected that it had reached the aortic valve. It then navigatedits tip around the perimeter of the valve annulus to the leak locationspecified from the preoperative imaging (c′_(i)→c_(i); FIG. 10, inset).

Switching between continuous and intermittent contact modes depends onthe robot recognizing the tissue type it is touching. We implemented thecapability for the catheter to distinguish the prosthetic aortic valvefrom blood and tissue using a machine learning classification algorithm.The classification algorithm first identified a collection of “visualwords,” which consisted of visual features shared between multipleimages in a set of prelabeled training images, and learned therelationship between how often these visual features occurred and whatthe image depicted—in this case, the prosthetic valve or blood andtissue.

Navigation on the annulus of the aortic valve to the location of a leakrequires two capabilities. The first is to maintain the appropriateradial distance from the center of the valve. The second is to be ableto move to a specified angular location on the annulus. For robustcontrol of radial distance, we integrated colored sutures into thebioprosthetic valve annulus that enable the navigation algorithm tocompute the tangent direction of the annulus (FIG. 12). Moving to aspecific angular location requires the robot to estimate its currentlocation on the annulus, to determine the shortest path around the valveto its target location, and to detect when the target has been reached.We programmed the robot to build a geometric model of the valve as itnavigates. On the basis of the estimated tangent direction of the valveannulus, as well as basic knowledge of the patient and robot position onthe operating table, the robot could estimate its clock face position onthe valve (FIG. 12). To account for valve rotation relative to the robotdue to variability in patient anatomy and positioning, we incorporatedradially oriented colored registration sutures spaced 120° apart. As thecatheter navigated along the annulus and detected the registrationsutures, it updated its valve model to refine the estimate of itslocation on the valve.

The algorithm inputs are consisted of the clock-face leak location andthe desired ventricular approach direction, also specified as aclock-face position. Starting from just inside the apex of the leftventricle, the catheter moved in the desired approach direction until itdetected tissue contact. It then switched to continuous contact mode andperformed wall following in the direction of the valve. When theclassifier detected the bioprosthetic valve in the haptic vision image,the controller switched to intermittent contact mode and computed theminimum distance direction around the annulus to the leak location basedon its initial map of the annulus. As the catheter moved around theannulus in this direction, its map was refined on the basis of thedetection of tangent and registration sutures. Once the leak locationwas reached, the robot controller acted to maintain its position at thislocation and sent an alert to the operator. Using joystick control, theoperator could then reposition the working channel over the leak asneeded, and then, the occluder could be deployed.

Evaluation and Results

FIG. 14A, FIG. 14B, FIG. 14C, and FIG. 14D depict navigation completiontimes from in vivo experiments. (A) Navigation from apex of the leftventricle to the aortic annulus (FIG. 10, a→b→c). (B) Circumnavigationof the entire aortic valve annulus (e.g., c′_(i)→c₁→c₂→c′₃→c′₂→c₃→c′₁;FIG. 10, inset). (C) Navigation from apex to paravalvular leak (FIG. 10,a→b→c′_(i)→c_(i)) (D) Deployment of vascular occluder. Red bars indicaterange, and dots denote outliers. P values computed as described in the“Statistical analysis” section further below.

The goal of the study was to investigate the feasibility of performingautonomous catheter navigation for a challenging intracardiac procedurein a preclinical porcine in vivo model. To perform this study, wedesigned and built a robotic catheter and haptic vision sensor. We alsodesigned and wrote control algorithms, enabling the catheter to navigateeither autonomously or under operator control. For our in vivoexperiments, we chose transapical paravalvular leak closure as ademonstration procedure and compared autonomous and operator-controllednavigation times with each other and with previous results using ahandheld catheter. For autonomous navigation, we also measured thedistance between the final position of the catheter tip and the actuallocation of the leak.

To evaluate the autonomous navigation algorithms, we performed in vivoexperiments comparing autonomous navigation with teleoperated (i.e.,joystick-controlled) robotic navigation. We also compared these twoforms of robotic navigation with manual navigation of a handheldcatheter. In all cases, the only sensing used consisted of the videostream from the tip-mounted endoscope, kinesthetic sensing of therobot/human, and force sensing of the human (handheld). At the end ofeach experiment, we opened the heart, examined the ventricular walls forbruising or other tissue damage, and found none.

We first compared success rate and navigation time for autonomousnavigation (FIG. 10, a→b→ci′) from the apex of the left ventricle to theaortic annulus (five animals, 90 trials), with teleoperated control(three animals, 9 trials) and with manual control (three animals, 13trials; FIG. 14A). Autonomous navigation consisted of first moving to awall of the ventricle specified by clock position (FIG. 12) and thenfollowing that wall to the valve using the continuous contact controlmode. Autonomous navigation was successful 99% of the time (89 of 90trials). Autonomous control was faster than teleoperated control andwith a smaller variance but slower than manual control.

Next, we investigated the ability of the controller to navigatecompletely around the valve annulus using the intermittent contact mode(e.g., c′₁→c₁→c₂→c′₃→c′₂→c₃→c′₁; FIG. 10, inset). This is substantiallymore challenging than what is required for paravalvular leak closurebecause, for leak closure, our algorithms enable the catheter to followthe ventricular wall in a direction that positions the catheter at anangular position on the valve that is close to the leak. For example, toreach ci in FIG. 10, the catheter could follow the path a→b→c′₁→c₁. Forcircumnavigation of the annulus, we compared autonomous control (threeanimals, 65 trials) with a handheld catheter (three animals, 3 trials)and with two forms of teleoperation (FIG. 14B). The first consisted ofstandard teleoperated control (one animal, 9 trials). The secondcorresponds to autonomous operator assistance (one animal, 10 trials).In the latter, the robot automatically controlled motion perpendicularto the plane of the valve to achieve a desired contact duty cycle,whereas the human operator manually controlled motion in the valveplane. Autonomous valve circumnavigation was successful 66% of the time(43 of 65 trials). Manual and teleoperated control had 100% successrates because the human operator, a clinician, could interpret andrespond to unexpected situations. For this task, teleoperation wasfaster than autonomous and manual navigation, with assistedteleoperation being the fastest (FIG. 14B). Autonomous control was theslowest, taking over twice as long as manual control.

We then compared controller performance for the complete paravalvularleak navigation task (FIG. 10; a→b→ci′→ci), in which the catheterstarted at the heart's apex, approached the ventricular wall in auser-provided direction, moved to the aortic annulus along theventricular wall, and then followed the annulus to the prespecified leakposition (five animals, 83 trials). We chose the direction along theventricular wall so that the catheter would arrive on the valve at apoint c′_(i), close to the leak c_(i), but such that it would still haveto pass over at least one registration marker to reach the leak.Autonomous navigation was successful in 95% of the trials (79 of 83)with a total time of 39±17 s compared with times of 34±29 s forteleoperation (three animals, 9 trials) and 31±27 s for manualnavigation (three animals, 13 trials) (see FIG. 14B). Note that forteleoperated and manual navigation, the operator was not required tofollow a particular path to a leak.

For autonomous navigation, we also evaluated how accurately the catheterwas able to position its tip over a leak. In the first three theexperiments, valve rotation with respect to the robot was estimated byan operator before autonomous operation. In the last two experiments,valve rotation was estimated by the robot based on its detection of theregistration sutures. The distance between the center of the cathetertip and the center of each leak was 3.0±2.0 mm for operator-basedregistration (three animals, 45 trials) and 2.9±1.5 mm for autonomousestimation (two animals, 38 trials) with no statistical differencebetween methods (P=0.8262, Wilcoxon rank sum). This error is comparablewith the accuracy to which a leak can be localized on the basis ofpreoperative imaging.

To ensure that autonomous navigation did not affect occluder delivery,we performed leak closure after autonomous, teleoperated, and manualnavigation. The time to close a leak was measured from the moment eitherthe robot or the human operator signaled that the working channel of thecatheter was positioned over the leak. Any time required by the operatorto subsequently adjust the location of the working channel was includedin closure time (FIG. 14D). Leak closure was successful in 8 of 11trials (autonomous navigation), 7 of 9 trials (teleoperation), and 11 of13 trials (manual navigation). The choice of navigation method producedno statistical difference in closure success or closure time.

FIG. 15 depicts a representative software development cycle. Insimulation, we replayed data from previous in vivo experiments toevaluate and debug software. New features were first implemented in thesimulator either to address previously identified in vivo challenges orto extend robot capabilities. New software was then tested in the exvivo model to check the desired functionality and to ensure codestability. Identified problems were addressed by iterating between insilico and ex vivo testing. New software features were then assessedwith in vivo testing. The design cycle was then completed by importingthe in vivo data into the simulator and evaluating algorithmperformance.

To develop and test our autonomous navigation algorithms, we implementeda development cycle composed of three steps: in silico simulation, exvivo experiments, and in vivo experiments (FIG. 15).

We created a simulation engine that can replay time-stamped data,comprising haptic vision images and robot trajectories, recorded duringin vivo cases. We used the simulation engine to implement new softwarefunctionality and to troubleshoot unexpected results from in vivoexperiments. After simulation, we tested new functionality on an ex vivomodel comprising an explanted porcine heart, pressurized using aperistaltic pump (Masterflex Pump, 115 VAC). We immobilized thepressurized heart using sutures to attach it to a fixture. On the basisof the outcome of the ex vivo tests, we either performed additionalsimulations to refine the software implementation or proceeded to invivo testing. This process was repeated iteratively for each algorithmas it was developed.

The software was executed on two PCs. One was used for catheter motioncontrol [Intel Core Quad CPU Q9450@2.66 GHz with 4-GB random-accessmemory (RAM)], whereas the second was used to acquire and process imagesfrom the haptic vision sensor (Intel Core i7-6700HQ CPU@2.6 GHz with16-GB RAM). The two computers exchanged information at runtime viatransmission control protocol/internet protocol. The motion controlcomputer received real-time heart rate data by serial port (Advisor,SurgiVet) and was also connected through universal serial bus to asix-DOF joystick (Touch, 3D Systems) that was used during teleoperatedcontrol of catheter motion. The motion control computer could executeeither the autonomous navigation algorithms or the joystick motioncommands. In either case, catheter tip motion commands were converted tosignals sent to the motor amplifiers of the catheter drive system.

The catheter control code converting desired catheter tip displacementsto the equivalent rotations and translations of the individual tubes waswritten in C++. The code was based on modeling the kinematics using afunctional approximation (truncated Fourier series) that was calibratedoffline using tip location data collected over the workspace. Thecalibrated functional approximation model had been previouslydemonstrated to predict catheter tip position more accurately (i.e.,smaller average and maximum prediction error) over the workspacecompared with the calibrated mechanics-based model. Catheter contactwith tissue along its length produced unmodeled and unmeasureddeformations that must be compensated for via tip imaging. Ahierarchical control approach was used to ensure that the desired tipposition was given a higher priority than the desired orientation ifboth criteria could not be satisfied simultaneously.

In Vivo Experiments

Interventional Procedure

We created a porcine paravalvular leak model by implanting a custombioprosthetic device (FIG. 12) into the aortic valve position in84.3±4.7 kg Yorkshire swine. The device was designed with three sewingring gaps evenly distributed around its circumference (120° apart) toproduce the areas of paravalvular leakage. The bio-prosthetic valveconsists of a titanium frame covered by a nonwoven polyester fabric. Apolypropylene felt sewing ring is sutured to the frame around theannulus. Suture is passed through this ring when the valve is sewn inplace inside the heart. Last, glutaraldehyde-fixed porcine pericardiumleaflets are sutured to the frame.

Animal care followed procedures prescribed by the Institutional AnimalCare and Use Committee. To implant the bioprosthetic valve, wepremedicated the swine with atropine (0.04 mg/kg intramuscularly),followed by Telazol (4.4 mg/kg) and xylazine (2.2 mg/kg intravenously),and we accessed the thoracic cavity through a median sternotomyincision. We acquired epicardial echocardiographic images to determinethe size of the valve to be implanted. Next, we initiatedcardiopulmonary bypass by placing purse-string sutures for cannulation,cross-clamping the aorta, and infusing cardioplegia solution to induceasystole. We incised the aorta to expose the valve leaflets, which werethen removed, and the artificial valve was implanted using nine 2-0ETHIBOND valve sutures supra-annularly. At this point, we closed bysuture the aortomy incision, started rewarming, and released the aorticcross-clamp. We maintained cardiopulmonary bypass to provide 35 to 50%of normal cardiac output to ensure hemodynamic and cardiac rhythmstability. The function of the implanted valve, as well as the leaklocations and sizes, were determined by transepicardial short- andlong-axis 2D and color Doppler echocardiography. Apical ventriculotomywas then performed, with previous placement of purse-string sutures tostabilize the cardiac apex for the introduction of the robotic catheter.The catheter was introduced through the apex and positioned such thatits tip was not in contact with the ventricular walls. All experimentsin a group were performed using the same apical catheter position.Throughout the procedure, we continuously monitored arterial bloodpressure, central venous pressure, heart rate, blood oxygenation,temperature, and urine output. At the end of the experiment, aeuthanasia solution was injected, and we harvested the heart forpostmortem evaluation.

Autonomous Navigation from Apex to Valve

We performed experiments on five animals. For each animal, navigationwas performed using three valve approach directions corresponding to 6o'clock (posterior ventricular wall), 9 o'clock (ventricular septalwall), and 12 o'clock (anterior ventricular wall) (FIG. 12). Of the 90total trials, the number performed in the 6, 9, and 12 o' clockdirections were 31, 32, and 27, respectively.

Autonomous Circumnavigation of Aortic Valve Annulus

Experiments were performed on three animals. In the first experiment, arange of contact duty cycles was tested, whereas in the latter twoexperiments, the contact duty cycle was maintained between 0.3 and 0.4.In all experiments, the tangential velocity was specified as 2 mm/sduring those periods when the tip was not in contact with the valve and0 mm/s when in contact.

Autonomous Navigation from Apex to Paravalvular Leaks

We performed experiments on five animals. As an initial step for allexperiments, we built a 3D spatial model of the valve by exploring thevalve with the catheter under operator control. We used this model,which is separate from the model built by the autonomous controller, tomonitor autonomous navigation. For three animals, we also used thismodel to estimate valve rotation with respect to the robot and providedthis estimate as an input to the autonomous navigation algorithm. In twoanimals, valve rotation was estimated autonomously on the basis of thevalve model built by the navigation algorithm and its detection ofregistration sutures.

In each experiment, navigation trials were individually performed foreach of the three leaks located at 2 o'clock (n=28), 6 o' clock (n=27),and 10 o'clock (n=28) (FIG. 12). For each leak location, we selected aclock direction to follow on the ventricular wall such that the catheterwould arrive at the valve annulus close to the leak but far enough awaythat it would have to pass over registration sutures to reach the leak.In general, this corresponded to approaching the valve at 11, 9, and 1o'clock to reach the leaks at 2 o'clock (clock-wise), 6 o'clock (counterclockwise), and 10 o'clock (counter clockwise), respectively. If weobserved that along these paths the annulus was covered by valve tissueor a suturing pledget, then we instructed the navigation algorithm toapproach the leak from the opposite direction. Note that the mitralvalve is located from 2 to 5 o'clock; the ventricular wall cannot befollowed in these directions to reach the aortic valve. Thus, in oneexperiment involving operator-specified valve registration, theclockwise-approach path was covered by tissue, and we chose to approachthe leak directly from the 2 o'clock direction rather than start fartheraway at 6 o'clock.

We designed the registration sutures to be 120° apart under theassumption to that valve rotation with respect to the robot would beless than ±60° from the nominal orientation. In one animal in whichvalve rotation was estimated autonomously, however, the rotation anglewas equal to 60°. In this situation, it is impossible for either man ormachine to determine whether the error is +60° or −60°. For theseexperiments, we shifted the approach direction for the leak at 6 o'clockfrom 9 to 8 o'clock so that the catheter would only see one set ofregistration sutures along the path to the leak. This ensured that itwould navigate to the correct leak.

Occluder Deployment

FIG. 16 depicts a representative occluder deployment system. Theoccluder, attached to a wire via a screw connection, is preloaded insidea flexible polymer delivery cannula. The delivery cannula is insertedthrough the lumen of the catheter into the paravalvular leak. A polymerdeployment tube is used to push the occluder out of the deliverycannula. Once positioned, the occluder is released by unscrewing thewire.

After navigation to the desired leak location, the operator took controlof the catheter and, as needed, centered the working channel over theleak. A three-lobed vascular occluder (AMPLATZER Vascular Plug II, AGAMedical Corporation), attached to a wire and preloaded inside a deliverycannula, was advanced ˜3 mm into the leak channel (FIG. 16). The cannulawas then withdrawn, allowing the occluder to expand inside the leakchannel. We then retracted the wire and robotic catheter until theproximal lobe of the occluder was positioned flush with the valveannulus and surrounding tissue. If positioning was satisfactory, thenthe device was released by unscrewing it from the wire. If not, then thedevice was retracted back into the delivery cannula and the procedurewas repeated as necessary.

In Vivo Calibration of Contact Duty Cycle Versus Maximum Tissue Force

FIG. 17A, FIG. 17B, and FIG. 17C illustrate in vivo calibration ofmaximum tissue force as a function of contact duty cycle. In particular,FIG. 17A illustrates a handheld instrument for the simultaneousmeasurement of tip contact force and contact duty cycle that combines ahaptic vision sensor with a force sensor. FIG. 17B depicts in vivomeasurements of the temporal variations in contact force as a functionof contact duty cycle on the aortic valve annulus. The insets showimages from the haptic vision sensor at three points in the cardiaccycle. Note that the minimum force value is not necessarily zero becausea small amount of tip contact with ventricular tissue can occur duringsystole when the valve moves away from the tip, but the ventriclecontracts around it. This white (septal) ventricular tissue can be seenon the left side of the rightmost inset. FIG. 17C depicts maximumcontact force measurements as a function of duty cycle. Average valuesof maximum force are linearly related to contact duty cycle for dutycycles in the range of 0.35 to 0.7.

To investigate the relationship between maximum contact force andcontact duty cycle, we designed a handheld instrument integrating hapticvision and force sensing (FIG. 17A). The haptic vision sensor is mountedon a stiff tube that is supported by two polymer sliding bearingsmounted inside the proximal handle. The proximal end of the shaft isconnected to a force sensor. An outer cannula encloses the sensing shaftand extends from the handle to about 6 cm from the sensing tip. When theinstrument is inserted into the apex of the heart, the outer cannula isin contact with the apical tissue, but the sensing tube is not. The gapbetween the outer cannula and sensing tube is filled with siliconegrease to prevent blood flow from the heart into the instrument whilegenerating minimal friction on the sensing tube. Calibration experimentsindicated that the friction due to the bearings and grease was less than±0.2 N.

We performed in vivo experiments in which we positioned the hapticvision sensor on the bioprosthetic valve annulus in locations where wecould be sure that the sensor was experiencing contact only on its tip.At these locations, we collected force, heart rate, and haptic visiondata (FIG. 17B). By manually adjusting instrument position along itsaxis, we were able to obtain data for a range of duty cycle values. Theimage and force data were collected at 46 Hz, and contact duty cyclebased on valve contact was computed at each sampling time using a datawindow of width equal to the current measured cardiac period (˜36images). To remove high-frequency components not present in the forcedata, we then filtered the computed duty cycle using a 121-sample movingaverage filter corresponding to ˜3.4 heartbeats. The filtered input dataand the output data (e.g., FIG. 17B) were then binned using duty cycleintervals of 0.05. Last, we computed the relationship between filteredcontact duty cycle and maximum applied force by averaging the maximumforces for each binned duty cycle value (FIG. 17C). We computed thePearson's coefficient as a measure of linear relationship between thecontact duty cycle and the maximum annular force. The Pearson'scoefficient was equal to 0.97, which indicates a strong linearrelationship. The plot of FIG. 17C indicates that the contact duty cyclerange of 0.35 to 0.45 that we used in most of our experimentscorresponded to a maximum force of 1.25 to 2.3 N.

Statistical Analysis

MATLAB (version R2017b) statistical subroutines were used to analyze thedata and perform all statistical tests. We compared time duration foreach navigation mode (i.e., handheld, teleoperated, and autonomous) forthe tasks of navigating from the apex to the aortic annulus, navigatingaround the valve annulus, and from the apex to the leak. We alsocompared occluder deployment times for each navigation mode. Groups,corresponding to different navigation modes, have unequal sample sizesand sample variances. We used Levene's test to evaluate equality ofvariances. With no evidence of normally distributed time duration andmore than two groups, we used the Kruskal-Wallis nonparametric test tocheck whether there are statistically significant time differences amonggroups. In experiments with statistical significance, we compared pairsof groups using the Mann-Whitney U test with Bonferroni correction. Dataless than Q1-1.5×IQR or greater than Q3+1.5×IQR, where the interquartilerange (IQR)=Q3−Q1, were considered outliers. Fisher's exact test wasused to compare success rates between different groups in the case ofparavalvular leak closure. Statistical significance was tested at the 5%confidence level (P<0.05).

Non-Contact Autonomous Navigation

In addition to or as an alternative to a force sensor, robotic system100 may comprise a distance sensor 420 in conjunction with imagingdevice 411 to facilitate autonomous navigation to the interventionalsite. Distance sensor 420 may enable robotic system 100 to navigate tothe interventional site using wall-following techniques, only withoutcontacting (or only intermittently contacting) anatomical features,medical implants, or medical instruments along the path.

Generally speaking, distance sensor 420 may constantly or intermittentlymeasure a distance and/or a direction between distance sensor 420 andnearby anatomical features, medical implants, or medical instruments,and the controller may utilize these distance and/or directionmeasurements to steer along a path that follows, but generally avoidscontact with, the wall. For example, in an embodiment, the controllermay be configured to steer the system so as to maintain a thresholddistance from the anatomical feature, medical implant, or medicalinstrument. This may improve safety, navigational accuracy, and speed.

As before, in various embodiments, the controller may be configured todetermine the system's location within the anatomy by processing the oneor more images captured by imaging device 411 using the previouslydescribed techniques. To the extent it is necessary to displace fluidbetween imaging device 411 and the anatomical feature, medical implant,or medical instrument in order to capture a suitable image, in variousembodiments, the controller may be configured to steer imaging device411 toward the anatomical feature, medical device, or medical instrumentand make contact therewith as previously described. Images may becaptured and processed frequently enough to confirm the location of thesystem in the body as it is advanced to the interventional site, whiledistance sensor 420 steers the system to follow the wall.

Distance sensor 420, in various embodiments, may include a light-baseddistance sensor (e.g., those emitting laser, infrared, or other light tomeasure distance), a sound-based distance sensor (e.g., those emittingultrasonic or other sound waves to measure distance), an energy-baseddistance sensor (e.g., radar and the like), or any other sensor suitablefor measuring a distance and/or a direction between distance sensor 420and the anatomical features, medical devices, and/or medical instrumentssituated a path to the interventional site.

Additionally or alternatively, in various embodiments, imaging device410 may serve as distance sensor 420. For example, in an embodiment inwhich imaging device 410 includes an image sensor 411, the controllermay process one or more images captured by image sensor 411 according totechniques suitable for estimating distance and/or direction based on arelative size of the anatomical feature, implanted device, or medicalinstrument in the image. For example, in non-contact wall-followingembodiments, in which image sensor 411 captures the image at a distance,the controller may be configured to compare the relative size of theanatomical feature, implanted device, or medical instrument in the imageto one or more reference images in which the distance is known, andestimate distance based on proportionality or other relationaltechniques (e.g., we know the implanted device is 1 cm wide and thatmeans X image pixels=1 cm). Generally speaking, in such embodiments, thesmaller the anatomical feature, implanted device, or medical instrumentappears in the image, the farther away it may be from image sensor 411,and vice versa. It should be recognized that non-contactvisible-spectrum image capture approaches may best be used in portionsof the anatomy in which fluid is not likely to obstruct the view ofimage capture device 411—for example, in air-filled passages in thelungs as opposed to in blood-filled passages in the heart. Likewise, asanother example, similar techniques may be used to estimate distanceand/or direction in ultrasound images in embodiments in which imagingdevice 410 includes an ultrasonic probe. Still further, distance and/ordirection could be estimated using known techniques in the art forassessing distances and/or directions from the ultrasonic probe inultrasound imagery. For example, since scale factors in ultrasoundimages are generally known, the controller may be configured to utilizeimage processing techniques to detect a boundary of the anatomicalfeature, implanted device, or medical instrument and then estimatedistance and/or direction based on the image. One of ordinary skill inthe art will recognize other approaches suitable for estimating distancein non-contact embodiments within the scope of the present disclosure.

Although the present disclosure and its advantages have been describedin detail, it should be understood that various changes, substitutionsand alterations can be made herein without departing from the spirit andscope of the disclosure as defined by the appended claims. Moreover, thescope of the present application is not intended to be limited to theparticular embodiments of the process, machine, manufacture, compositionof matter, means, methods and steps described in the specification. Asone of ordinary skill in the art will readily appreciate from thedisclosure, processes, machines, manufacture, compositions of matter,means, methods, or steps, presently existing or later to be developedthat perform substantially the same function or achieve substantiallythe same result as the corresponding embodiments described herein may beutilized according to the present disclosure. Accordingly, the appendedclaims are intended to include within their scope such processes,machines, manufacture, compositions of matter, means, methods, or steps.

What is claimed is:
 1. A robotic system, comprising: a robotic cathetersteerable by a motorized drive system; an imaging device positioned on adistal end of the robotic catheter; and a controller configured to:process one or more images captured by the imaging device to identify ananatomical feature, implanted device, or medical instrument in the oneor more images, estimate a location of the imaging device in the bodybased on the identified anatomical feature, implanted device, or medicalinstrument, determine, based on the estimated location of the imagingdevice, a direction in which to steer the robotic catheter foradvancement towards an interventional site, and monitor at least one of(i) a stream of images captured by the imaging device and (ii) force ordistance measurements captured by the imaging device or by a sensorproximate the imaging device, to adjust the direction in which to steerthe robotic catheter during advancement towards the interventional site.2. The robotic system of claim 1, wherein the robotic catheter iscomprised of two or more concentric tubes.
 3. The robotic system ofclaim 1, wherein the motorized drive system is operable to rotate andtranslate the robotic catheter.
 4. The robotic system of claim 1,wherein the imaging device includes one of an image sensor, a camera, anultrasonic probe, or other device configured to capture the one or moreimages.
 5. The robotic system of claim 1, wherein the controller isconfigured to adjust the direction in which to steer the roboticcatheter so as to maintain constant or intermittent contact with ananatomical feature during advancement towards the interventional site.6. The robotic system of claim 1, wherein the imaging device includes asurface configured to displace bodily fluid from a contact interfacebetween the imaging window and the anatomical feature, the implanteddevice, or the medical instrument.
 7. The robotic system of claim 6,wherein the imaging device includes an imaging window covering theimaging device, and wherein the imaging window includes a surfaceconfigured to displace bodily fluid from a contact interface between theimaging window and the anatomical feature, implanted device, or medicalinstrument.
 8. The robotic system of claim 1, wherein processing one ormore images captured by the imaging device includes comparing the one ormore of the captured images to representative images of one or moreanatomical features, implanted devices, or medical instruments presentalong a pathway to a interventional site.
 9. The robotic system of claim1, wherein estimating a location of the imaging device in the bodyincludes identifying the location of the identified anatomical feature,implanted device, or medical instrument in an anatomical model.
 10. Therobotic system of claim 1, wherein determining the direction in which tosteer the robotic catheter for advancement towards a interventional siteincludes determining a vector between the estimated location of theimaging device and the interventional site using an anatomical model.11. The robotic system of claim 9, wherein the anatomical model isnon-dimensional.
 12. The robotic system of claim 1, wherein monitoring astream of images captured by the imaging device to adjust the directionin which to steer the robotic catheter includes identifying whether theimaging device is contacting the anatomical feature, implanted device,or medical instrument based on whether at least a portion of an image ofthe stream of images is unobstructed by bodily fluid.
 13. The roboticsystem of claim 1, wherein monitoring a force measurement to adjust thedirection in which to steer the robotic catheter includes determiningwhether the force measurement is substantially non-zero.
 14. The roboticsystem of claim 1, wherein the controller is further configured toestimate a contact force between the imaging device and the anatomicalfeature based on how much of the one or more images is unobstructed bybodily fluid.
 15. The robotic system of claim 14, wherein the controlleris further configured to use the estimated contact force or the forcemeasurement to avoid generating unsafe contact forces between theimaging device and the anatomical feature, the implanted device, or themedical instrument.
 16. The robotic system of claim 1, wherein thecontroller is further configured to estimate an orientation of theimaging device relative to the anatomical feature based on adistribution of the contacting surface area with respect to a center ofthe image.
 17. The robotic system of claim 1, wherein monitoring adistance measurement to adjust the direction in which to steer therobotic catheter includes determining a distance to the anatomicalfeature, implanted device, or medical instrument.
 18. The robotic systemof claim 17, wherein the controller is configured to adjust thedirection in which to steer the robotic catheter so as to avoid contactwith an anatomical feature during advancement towards the interventionalsite.
 19. A system, comprising: a catheter; an imaging device positionedon the distal end of the catheter; and a processor configured to processone or more images captured by the imaging device to automaticallyperform at least one of the following: identify an anatomical feature,implanted device, or medical instrument contacted by the imaging deviceby comparing the one or more captured images to representative images ofone or more anatomical features, implanted devices, or medicalinstruments present along a pathway to a interventional site, andestimate a contact force between the imaging device and the anatomicalfeature, implanted device, or medical instrument based on how much ofthe one or more images is unobstructed by bodily fluid.
 20. The systemof claim 19, wherein the catheter is manually steered by a person. 21.The system of claim 19, wherein the processor is further configured toestimate a location of the imaging device in the body based on theidentified anatomical feature, implanted device, or medical instrument.22. The system of claim 21, wherein estimating a location of the imagingdevice in the body includes identifying the location of the identifiedanatomical feature, implanted device, or medical instrument in ananatomical model.
 23. The system of claim 19, wherein the processor isfurther configured to determine, based on the estimated location of theimaging device, a direction in which to steer the catheter foradvancement towards a interventional site.
 24. The system of claim 23,wherein determining the direction in which to steer the catheter foradvancement towards a interventional site includes determining a vectorbetween the estimated location of the imaging device and theinterventional site using an anatomical model.
 25. The system of claim22, wherein the anatomical model is non-dimensional.
 26. The system ofclaim 19, wherein the processor is further configured to use theestimated contact force to avoid generating unsafe contact forcesbetween the imaging device and the anatomical feature, implanted device,or medical instrument.
 27. The system of claim 19, wherein the processoris configured to display, on a display device, one or more of: the oneor more captured images, the estimated location of the imaging device,the direction in which to steer the catheter for advancement towards theinterventional site, and the estimated contact force.